Diagnostic Accuracy of Mental Health Screening Tools After Mild Traumatic Brain Injury

Key Points Question Can self-report screening tools accurately detect a major depressive episode, anxiety disorder, and posttraumatic stress disorder (PTSD) after mild traumatic brain injury (mTBI)? Findings In this diagnostic study with 499 participants with mTBI, the Patient Health Questionnaire–9, Generalized Anxiety Disorder–7, and Primary Care PTSD Screen for the Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) screening tools had acceptable diagnostic accuracy; the Generalized Anxiety Disorder–7 accurately identified not only anxiety disorders but also PTSD. In patients with persistent postconcussive symptoms, specificity was lower and mental health disorders were more common. Meaning These findings suggest that brief self-report tools can reliably screen for mental health disorders after mTBI.


Introduction
2][3][4][5] The prevalence rates for mental health conditions 3 to 12 months after mTBI are 17% to 27% for depressive disorders, 1,6,7 11% to 24% for anxiety disorders, 6,8,9 and 10% to 21% for PTSD. 3,5,10,11Mental health disorders likely exacerbate persistent postconcussion symptoms (PPCS) [11][12][13][14][15] and contribute to the substantial rate (30%-50%) of chronic disability after mTBI. 12,16Outcomes from mTBI could be optimized by proactively monitoring for new or worsened mental health disorders (eg, with self-report screening scales) and initiating mental health treatment. 15ntal health disorders can be efficiently detected in primary care with self-report screening tools such as the Patient Health Questionnaire-9 (PHQ-9) for a current major depressive episode (MDE), 17,18 General Anxiety Disorder-7 (GAD-7) for anxiety disorders, 19 and Primary Care PTSD Screen for the DSM-5 (Diagnostic and Statistical Manual of Mental Disorders [Fifth Edition]) (PC-PTSD-5) for PTSD.20,21 The PHQ-9 and GAD-7 are among the most extensively studied and widely used mental health screens in primary care.Meta-analyses have demonstrated good diagnostic accuracy for the PHQ-9 in identifying MDE, with no substantial differences between investigated subgroups (eg, sex and recruitment setting), [22][23][24] and that the GAD-7 can detect generalized anxiety disorder and other anxiety disorders.25 Preliminary evidence suggests that these tools have comparable diagnostic accuracy in mTBI, at least in outpatient [26][27][28] and hospitalized samples with trauma-related intracranial lesions, 29 but establishing diagnostic accuracy in larger and more generalizable samples with mTBI is needed (only 15% of adults with mTBI have traumarelated intracranial lesions).30 It is also unclear whether optimal cutoff scores differ for patients with mTBI, especially in the presence of PPCS.Sleep disturbance, fatigue, concentration difficulties, and mood symptoms are transdiagnostic, that is, common to both PPCS and mental health disorders, 14,31 raising concerns about the performance of self-report screening tools in mTBI.
The PC-PTSD-5 was originally developed for use with military service members and veterans. 32 appears to be useful in primary care 20 but has not been evaluated as extensively as the PHQ-9 or GAD-7, nor has it been previously studied in mTBI.Its appeal in this population is that it only queries core PTSD symptoms that are specific to PTSD (ie, those that do not overlap with PPCS).Evidence from primary care studies suggests that the GAD-7 may be sensitive to PTSD, 25,[33][34][35][36] but this has not been evaluated within the context of mTBI.
Clinical practice guidelines for mTBI recommend routine screening for mental health disorders, 37,38 but insufficient evidence is available to guide implementation of this recommendation.The present study evaluated the diagnostic accuracy of the PHQ-9, GAD-7, and PC-PTSD-5 screening tools against a criterion standard structured diagnostic interview identification of MDE, anxiety disorders, and PTSD.Secondary aims were to compare diagnostic accuracy in those with and without PPCS, to determine whether the GAD-7 can simultaneously screen for PTSD as well as a PTSD-specific screening tool (the PC-PTSD-5), and to determine whether the combination of the GAD-7 and PC-PTSD-5 can optimize PTSD screening.

Methods
This diagnostic study was a secondary analysis of a cluster randomized clinical trial that analyzed whether a clinical practice guideline implementation tool designed to support early detection of mental health disorders after mTBI could lower the risks of mental health complications. 39,40The study was reviewed and approved by the University of British Columbia Clinical Research Ethics Board, and all participants gave informed consent to participate.We followed the Standards for Reporting of Diagnostic Accuracy (STARD) reporting guidelines.

JAMA Network Open | Psychiatry
Diagnostic Accuracy of Mental Health Screening Tools After Mild TBI

Participants
Participants were recruited from February 1, 2021, to October 15, 2022.A total of 537 participants with an mTBI were recruited from 6 emergency departments and 2 urgent care centers in the Greater Vancouver area in British Columbia, Canada.Participants were included if they were aged 18 to 69 years, presented to care within 72 hours, met World Health Organization Neurotrauma Task Force criteria for mTBI (eMethods in Supplement 1), 41 were fluent in English, had their primary residence in British Columbia, and designated a general practitioner or a walk-in clinic where they would seek follow-up care.Individuals with preexisting unstable or serious illnesses were excluded.
Participant race and ethnicity were self-reported and included the following categories: Arab; Black; Caribbean; East or Southeast Asian; Fijian; First Nation, Inuit, or Métis; Latinx; Middle Eastern; South Asian; White; and preferred not to answer.These data were collected to provide information about the generalizability of the study results.
Participants consented by completing an online form through REDCap (Research Electronic Data Capture; Vanderbilt University). 42Details about the recruitment procedures are available in the trial protocol paper. 39

Measures
Patient Health Questionnaire-9 The PHQ-9 17 is a 9-question assessment developed to measure depressive symptoms.Participants rate how frequently they experienced symptoms in the past 2 weeks, from 0 (not at all) to 3 (nearly every day).The total score ranges from 0 to 27.A range of cutoffs have been proposed, with total score of 10 or more being most common.Additionally, 3 algorithms have been considered for identifying MDEs that require the endorsement of at least 1 cardinal symptom (depression or anhedonia) and 1 of the following: a score of at least 10, 28,29 5 or more symptoms rated as experienced more than half of the days (suicidal ideation is counted if endorsed with any frequency), 17,29 and 5 or more symptoms rated as experienced several days (eTable 1 in Supplement 1). 29

Generalized Anxiety Disorder-7
The GAD-7 is a 7-question screening tool that measures anxiety symptoms. 19Participants rate how frequently they have experienced symptoms over the past 2 weeks from 0 (not at all) to 3 (nearly every day).The total scores range from 0 to 21.The cutoff scores of at least 7 27 and at least 10 19 have been proposed for the indication of an anxiety disorder (eTable 1 in Supplement 1).In addition to these accepted cutoff scores, we also investigated cutoff scores ranging from 5 to 15 (eTables 10, 12, and 14 in Supplement 1).The diagnostic accuracy of the GAD-7 for generalized anxiety disorder is provided in eTables 16 to 21 in Supplement 1.

Primary Care PTSD Screen for DSM-5
The PC-PTSD-5 is a 5-question instrument. 32First, participants are asked about their lifetime trauma exposure.If they indicate that they have had no exposure to an unusual, frightening, traumatic, or horrific event, their score is 0. If they have been exposed to such an event, participants respond 0 (indicating no) or 1 (indicating yes) to 5 questions about symptoms they may be experiencing, with the total score ranging from 0 to 5. Cutoff scores of at least 3 21,32 and at least 4 20 have been proposed for indication of PTSD (eTable 1 in Supplement 1).

Rivermead Post-Concussion Questionnaire
The Rivermead Post-Concussion Symptoms Questionnaire (RPQ) consists of 16 symptoms.Each symptom is rated on a scale from 0 (indicating not experienced at all) to 4 (indicating a severe problem). 43

Mini-International Neuropsychiatric Interview
The MINI is a structured diagnostic interview that is designed to assess the most common psychiatric disorders in clinical and research settings. 44We administered 8 modules assessing MDEs, anxiety disorders (ie, panic disorder, agoraphobia, social anxiety disorder, specific phobia, generalized anxiety disorder, or obsessive-compulsive disorder), and PTSD.Version group supervision with the psychologist to discuss complex cases and resolve uncertain MINI coding.
To ensure standardized administration, assessors were periodically audited (by the supervising psychologist observing an assessment live) to prevent drift from standardized administration.
After administering the MINI, participants were e-mailed a REDCap 42 online survey that included the PHQ-9, GAD-7, PC-PTSD-5, and the RPQ.The outcome assessors were blind to the results of the screening questionnaires.

Statistical Analysis
Participant characteristics and responses on the screening questionnaires were described in terms of central tendency.Missing responses on the screening tools were replaced with the mean score of the questionnaire if participants missed fewer than 20% of items (26 participants [5.2%] missed 1 to 3 responses on the RPQ, 19 [3.8%] missed 1 response on the PHQ-9, 8 [1.6%] missed 1 response on the GAD-7, and 3 [0.6%]missed 1 response on the PC-PTSD-5).
We derived the receiver operating characteristics curve to determine the area under the curve (AUC) of the total scores of each screening questionnaire for the diagnosis of MDE (based on meeting criteria for a current MDE), any anxiety disorder (Ն1), or PTSD according to the results of the MINI.
An AUC of 1.0 describes perfect discrimination. 45Additionally, we investigated the sensitivity, specificity, positive predictive values (PPV), negative predictive values (NPV), and likelihood ratios at various cutoff scores.Finally, we determined 95% CIs by using the exact binomial confidence interval method. 46 assessed the diagnostic accuracy in the overall sample and separately for those with and without PPCS.Patients were categorized as having PPCS present or absent based on their endorsement of symptoms on the RPQ that met the International and Statistical Classification of Diseases, Tenth Revision, category C criteria for postconcussional syndrome (F07.81), as in previous studies. 47,48The RPQ assesses for categories I through IV (physical, emotional, and cognitive symptoms and poor sleep) of postconcussional syndrome, and participants were categorized as having PPCS present if they endorsed at least 1 symptom as moderate to severe (ie, item scoring Ն3) in at least 3 categories. 49 determine whether the diagnostic accuracy of the GAD-7 is equivalent to the PC-PTSD-5 when detecting symptoms of PTSD, we compared their AUCs, sensitivity, and specificity at various cutoffs.Additionally, we performed a multivariable logistic regression to explore whether combining the questionnaires' total scores would improve the ability to detect PTSD, measured by using likelihood ratio tests.
The analyses were performed using R packages pROC, version 1.
For each cutoff, the positive likelihood ratios were greater than 2.00 and the negative likelihood ratios were less than 1.00.Additionally, the NPVs were 0.88 or greater, and the PPVs were 0.78 or less.Other cutoffs can be found in eTable 4 in Supplement 1.

Posttraumatic Stress Disorder
The AUC for the PC-PTSD-5 in the overall sample was 0.80 (95% CI, 0.72-0.87).A cutoff score of at least 3 had a sensitivity of 0.67 (95% CI, 0.52-0.80)with a specificity of 0.87 (95% CI, 0.84-0.90)(Table 3).A cutoff score of at least 4 yielded a poor sensitivity of 0.56 (95% CI, 0.41-0.71)with a specificity of 0.92 (95% CI, 0.90-0.95)(Table 3).The cutoff with the best balance between sensitivity and specificity, where both indices were at least 0.70, was a total score of at least 2 (eTable 23 in Supplement 1).The positive likelihood ratios for each threshold were greater than 2.00 and the negative likelihood ratios were less than 0.70.Additionally, the NPVs were greater than 0.90, and the PPVs were less than 0.60.

JAMA Network Open | Psychiatry
Diagnostic Accuracy of Mental Health Screening Tools After Mild TBI (χ 2 1 = 26.61;P < .001) or the PC-PTSD-5 (χ 2 1 = 21.35;P < .001)alone, showing that the use of both questionnaires was better at identifying PTSD, as expected (eTable 31 in Supplement 1 for the multivariable regression model).

Diagnostic Accuracy in Patients With mTBI and PPCS Present or Absent
The AUCs were lower in the PPCS-present subgroup (Ն0.75) compared with the PPCS-absent subgroup (Ն0.76) and overall sample (Ն0.80; difference, 0.01-0.13percentage points) (Figure , A-C, and eTable 32 in Supplement 1 provide a comparison of AUC values for each questionnaire).Across cutoffs, sensitivity was higher (difference, 14-33 percentage points) and specificity was lower (difference, 5-65 percentage points) in the PPCS-present subgroup compared with the PPCS-absent subgroup and overall sample.However, PPV remained high for the PPCS-present subgroup, as it was offset by a higher base rate of mental health disorders (ie, 3 to 5 times higher).Also of note, the cutoff of at least 5 symptoms with a total score of at least 2 on the PHQ-9 was relatively robust to PPCS status, maintaining reasonable specificity (0.74 [95% CI, 0.62-0.84]).Relative to the PHQ-9 and GAD-7, the diagnostic accuracy of the PC-PTSD-PC was less affected by PPCS status (Figure, C vs A and B and Table 4).

Discussion
The present study supports the diagnostic accuracy of the PHQ-9 for detecting MDE, GAD-7 for detecting anxiety disorders, and PC-PTSD-5 for detecting PTSD in patients with mTBI.The overall    45 Similar to previous studies in primary care patients and TBIs of all severities, the PHQ-9 had outstanding accuracy (Ն0.90). 17,29,4533,42 Our study strengthens the evidence for the PHQ-9 and GAD-7 in a large and more representative sample with mTBI than previous studies and supports the diagnostic accuracy of the PC-PTSD-5.
When investigating previously recommended cutoffs for each questionnaire, we found some deviations in sensitivity and specificity.,32 To our knowledge, this is the first study to evaluate the diagnostic accuracy of the PHQ-9, GAD-7, and PC-PTSD-5 in adults with vs without PPCS.The AUC values were lower but still acceptable (Ն0.75) in participants with PPCS present.Despite reduced specificity in the PPCS-present subgroup, PPV was similar in the PPCS-present subgroup and the full sample because the base rate of mental health disorders was much higher in this group (relative risk, 2.96-10.97).In other words, a positive mental health screen result was associated with comparable likelihood of mental health  the same cutoffs could be used for all patients with mTBI.The co-occurrence of PPCS and mental health disorders is not surprising because of their symptom overlap, but also because PPCS can be distressing and, in turn, emotional distress can exacerbate PPCS. 12,54though the PHQ-9 total score cutoff of at least 10 and the algorithm of 5 or more symptoms with a score of at least 2 performed similarly in the full sample, the latter was more robust in the PPCS-present subgroup, providing a reason to favor it in specialty concussion clinic settings.The relatively simple and familiar cutoff of at least 10 could be used in primary care with only a modest loss of PPV.Further, because PPCS and mental health symptoms overlap, a detailed clinical assessment is needed to disentangle which symptoms are attributable to a mental health diagnosis, such as by querying the onset and course to determine whether they align better with that diagnosis.

JAMA Network Open | Psychiatry
Our analyses also revealed that the GAD-7 was slightly better at detecting the presence of PTSD compared with the PC-PTSD-5.The AUC for the GAD-7 was greater than 0.80, similar to a previous study of patients seen in primary care settings. 33Combining both questionnaires (PC-PTSD-5 and GAD-7) compared with each questionnaire alone was associated with a modest increase in the ability to detect PTSD (difference in AUC, 0.05; P < .001),but was likely not clinically meaningful.Using only the GAD-7 to screen for both anxiety disorders and PTSD is most efficient.However, if a high specificity (>0.85) is important, using the PC-PTSD-5 with a higher cutoff-or combining both questionnaires-may be more appropriate.
7.0.2 of the MINI is designed to align with DSM-5. 31Procedure At 12 weeks post injury, outcome assessors (ie, graduate students of clinical psychology or rehabilitation science) administered the MINI through a video-conferencing platform under the supervision of a registered psychologist (N.D.S.) to eligible participants.The outcome assessors were certified in Adult Standard MINI 7.0.2Training provided by the publisher and participated in weekly Abbreviations: GAD-7, Generalized Anxiety Disorder-7; MINI, Mini-International Neuropsychiatric Interview; PC-PTSD-5, Primary Care PTSD (Posttraumatic Stress Disorder) Screen for DSM-5 (Diagnostic and Statistical Manual of Mental Disorders [Fifth Edition]); PHQ-9, Patient Health Questionnaire-9; PPCS, persistent postconcussive symptoms; RPQ, Rivermead Post-Concussion Symptoms Questionnaire. a Higher scores indicate more frequent symptoms.b Higher scores indicate more symptoms experienced.c Higher scores indicate more severe problems.

Figure .
Figure.Receiver Operating Characteristics Curves for Each Screening Tool in the Overall Study Group and Subgroups With Persistent Postconcussive Symptoms (PPCS) Present and Absent

Table 1 .
Sociodemographic and Injury Characteristics of Participants b Participants were able to select more than 1 response.cOneparticipant (0.2%) did not report educational level.

Table 2 .
Prevalence of Disorders and Distributions of Screening Questionnaires
Downloaded from jamanetwork.comby guest on 07/26/2024 diagnosis in the PPCS-present subgroup and full sample, across cutoffs.This finding suggests that

Table 4 .
Diagnostic Accuracy for the Subgroups With PPCS Present and Absent Recommended Cutoff Scores for the Screening Tools eTable 2. Severity of Depression According to PHQ-9 eTable 3. PHQ-9 Diagnostic Accuracy for Overall Sample (N = 499) eTable 4. True and False Positive and Negative Rates for Each Cutoff for PHQ-9 in Overall Sample (N = 499) True and False Positive and Negative Rates for Each Cutoff for PHQ-9 for Sample With PPCS Absent (n = 341) eTable 9. Severity of Anxiety According to GAD-7 eTable 10.Diagnostic Accuracy of GAD-7 to Diagnose at Least 1 Anxiety Disorder for Overall Sample (N = 499) eTable 11.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose at Least 1 Anxiety Disorder for Overall Sample (N = 499) eTable 12. Diagnostic Accuracy of GAD-7 to Diagnose at Least 1 Anxiety Disorder for Sample With PPCS Present (n = 158) eTable 13.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose at Least 1 Anxiety Disorder for Sample With PPCS Present (n = 158) eTable 14.Diagnostic Accuracy of GAD-7 to Diagnose at Least 1 Anxiety Disorder for Sample With PPCS Absent (n = 341) eTable 15.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose at Least 1 Anxiety Disorder for Sample With PPCS Absent (n = 341) eTable 16.Diagnostic Accuracy of GAD-7 to Diagnose Generalized Anxiety Disorder for Overall Sample (N = 499) eTable 17.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose at Least 1 Anxiety Disorder for Overall Sample (N = 499) eTable 18. Diagnostic Accuracy of GAD-7 to Diagnose Generalized Anxiety Disorder for Sample With PPCS Present (n = 158) eTable 19.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose Generalized Anxiety Disorder for Sample With PPCS Present (n = 158) eTable 20.Diagnostic Accuracy of GAD-7 to Diagnose Generalized Anxiety Disorder for Sample With PPCS Absent (n = 341) eTable 21.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose Generalized Anxiety Disorder for Sample With PPCS Absent (n = 341) eTable 22. Severity of PTSD According to PC-PTSD-5 eTable 23.Diagnostic Accuracy of PC-PTSD-5 for Overall Sample (N = 499) eTable 24.True and False Positive and Negative Rates for Each Cutoff for PC-PTSD-5 for Overall Sample (N = 499) eTable 25.Diagnostic Accuracy of PC-PTSD-5 for Sample With PPCS Present (n = 158) eTable 26.True and False Positive and Negative Rates for Each Cutoff for PC-PTSD-5 for Sample With PPCS Present (n = 158) eTable 27.Diagnostic Accuracy of PC-PTSD-5 for Sample With PPCS Absent (n = 341) eTable 28.True and False Positive and Negative Rates for Each Cutoff for Sample With PPCS Absent (n = 341) eTable 29.Diagnostic Accuracy of GAD-7 to Diagnose PTSD for Overall Sample (N = 499) eTable 30.True and False Positive and Negative Rates for Each Cutoff for GAD-7 to Diagnose PTSD for Overall Sample (N = 499) eTable 31.Multivariable Regression Model eTable 32.AUC Comparison for Each Screening Questionnaire in the Overall, PPCS Present, and PPCS Absent Abbreviations: GAD-7, Generalized Anxiety Disorder-7; LR, likelihood ratio for positive or negative diagnostic test result; NPV, negative predictive value; PC-PTSD-5, Primary Care PTSD (Posttraumatic Stress Disorder) Screen for DSM-5 (Diagnostic and Statistical Manual of Mental Disorders [Fifth Edition]); PHQ-9, Patient Health Questionnaire-9; PPCS, persistent postconcussive symptoms; PPV, positive predictive value.a Indicates at least 1 cardinal symptom endorsed.b Suicidal ideation is counted if present regardless of duration.